Skip to content

fix(aws-dynamodb): paginate Query to avoid undercounting large result sets - #7912

Open
vidigoat wants to merge 2 commits into
kedacore:mainfrom
vidigoat:fix/aws-dynamodb-query-pagination
Open

fix(aws-dynamodb): paginate Query to avoid undercounting large result sets#7912
vidigoat wants to merge 2 commits into
kedacore:mainfrom
vidigoat:fix/aws-dynamodb-query-pagination

Conversation

@vidigoat

Copy link
Copy Markdown

Description

The AWS DynamoDB scaler's GetQueryMetrics issued a single dynamodb.Query call and returned that response's Count:

res, err := s.dbClient.Query(ctx, &dimensions)
...
return float64(res.Count), nil

A DynamoDB Query returns at most 1 MB of data per call and the result set can span multiple pages, signalled by a non-nil LastEvaluatedKey on the response. Because the scaler never followed that token, for any query whose matching items exceed a single page it only counted the first page and reported a Count far lower than the real number of matching items. This undercounts the metric and causes the workload to be under-scaled (the more items match, the worse the gap).

This is the same pagination pattern the AWS DynamoDB Streams scaler already uses (LastEvaluatedShardId).

Fix

Use dynamodb.NewQueryPaginator (which accepts the existing dynamodb.QueryAPIClient) to walk every page via LastEvaluatedKey and sum Count across all pages. Single-page results are unaffected, so this is non-breaking.

Tests

Added TestDynamoGetQueryMetricsPaginated: the mock returns two pages (4 items with a LastEvaluatedKey, then 3 items). It fails before the fix (returns 4) and passes after (returns 7). All existing DynamoDB scaler tests continue to pass.

Checklist

  • Tests have been added
  • Changelog has been updated and is aligned with our changelog requirements
  • Commits are signed with Developer Certificate of Origin (DCO)

Fixes #

… sets

A DynamoDB Query returns at most 1 MB of data per call and the response can
span multiple pages via LastEvaluatedKey. GetQueryMetrics only issued a single
Query and returned that page's Count, so for result sets larger than one page
the scaler undercounted the matching items and under-scaled the workload.

Follow the pagination token with dynamodb.NewQueryPaginator and sum Count across
all pages. Single-page results are unaffected. Adds a regression test that
returns two pages and asserts the counts are summed.

Signed-off-by: Vidit Patankar <vidit.patankar16@gmail.com>
@vidigoat
vidigoat requested a review from a team as a code owner July 12, 2026 08:22
@github-actions

Copy link
Copy Markdown

Thank you for your contribution! 🙏

Please understand that we will do our best to review your PR and give you feedback as soon as possible, but please bear with us if it takes a little longer as expected.

While you are waiting, make sure to:

  • Add an entry in our changelog in alphabetical order and link related issue
  • Update the documentation, if needed
  • Add unit & e2e tests for your changes
  • GitHub checks are passing
  • Is the DCO check failing? Here is how you can fix DCO issues

Once the initial tests are successful, a KEDA member will ensure that the e2e tests are run. Once the e2e tests have been successfully completed, the PR may be merged at a later date. Please be patient.

Learn more about our contribution guide.

@snyk-io

snyk-io Bot commented Jul 12, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@keda-automation
keda-automation requested a review from a team July 12, 2026 08:22
Signed-off-by: Vidit Patankar <vidit.patankar16@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant